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COHOMOLOGY OF LINE BUNDLES ON THE FLAG 
VARIETY FOR TYPE G2 

HENNING HAAHR ANDERSEN AND KANEDA MASAHARU 



O 

Cs| ' Abstract. In the case of a simple algebraic group G of type G2 

Oj. over a field of characteristic p > we study the cohomology mod- 

D ' ules of line bundles on the flag variety for G. Our main result is 

C^ , a complete determination of the vanishing behavior of such coho- 

mology in the case where the line bundles in question are induced 
by characters from the lowest p^-alcoves. 

When Uq is the quantum group corresponding to G whose pa- 
r I ' , rameter g is a complex root of unity of order prime to 6 we give a 



complete (i.e. covering all characters) description of the vanishing 
behavior for the corresponding quantized cohomology modules. 



CN ■ 1. Introduction 

> 

in 

(^ ! Throughout this paper G will denote an almost simple algebraic 

cn I group of type G2 over a field of characteristic p > 5. We shall give 

C^ ■ a complete description of the vanishing behavior of the cohomology of 

those line bundles on the fiag variety for G which correspond to weights 
from the lowest p^-alcoves in all the Weyl chambers for G. To achieve 
this we shall need almost all available methods that we know for such 
-^ ■ computations. 

5t , The strategy we use will apply to other types as well but a similar 

description for higher ranks seems out of reach. Our computations 
also give some information about the G-structure of the cohomology 
modules but the full story here is still open. By an illustrating example 
(see Section 8) we demonstrate that our present techniques are not 
enough to handle the case where the weights lie outside the p^-alcoves. 
On the other hand, we can handle all weights for the corresponding 
quantized G2 situation at complex roots of unity, see below and Section 
9. 
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Our assumption p > 5 is necessary in order for alcoves to contain 
integral weights in their interior. We need this in order to apply trans- 
lation functors effectively. However, many parts of our methods apply 
for p < 5 as well but we leave it to the reader to formulate and check 
the statements in these cases (for instance when p = 2 the only weight 
in the lowest p^-range is —p so there is nothing to check in that case). 
Our results are given in terms of figures of alcoves decorated with those 
numbers i for which there is an extra non-vanishing i-th cohomology 
of the line bundles corresponding to the weights of the alcove. To see 
the patterns in these figures we need to take p a bit larger, p > 17, 
although our proofs work for all p > 5! The figures for smaller p will 
be a subset of our figures. We leave to the reader to do the appropriate 
cuts. 

Let Uq denote the quantum group of type G2 with q a complex root 
of unity of order /. We assume that / is prime to 6. Then there are 
cohomology modules for Uq (derived functors of the induction functor 
from a Borel subalgebra of Uq) which are quantized counterparts of the 
above line bundle cohomology. Our techniques apply to this case as 
well. Moreover, the modular phenomena of 'higher alcoves' (for the 
powers of p) are not present in this case. Hence we are able to give the 
complete vanishing behavoir of quantized line bundle cohomology for 
type G2. 

The questions studied in this paper go back to the final section of 
|A81] . Here the first author sketched a description of the vanishing 
behavior of line bundle cohomology on the flag varieties for all rank 
2 groups. However, as pointed out by the second author there are 
some errors in the statements and J. Humphreys pinpointed in [H] 
some specific inaccuracies in the G2-ca.se. In the survey article |A07j 
some of this was repaired but when we needed this kind of results in 
connection with our work |AK] we decided to carefully go through all 
computations. The outcome of this was recorded in Appendix B of 
|AK] where we referred to a preliminary version of the present paper 
for details. With a few modifications we follow the 6 step program 
outhned in this appendix. 

We are grateful to J. E. Humphreys for many discussions over the 
years on these issues and for some detailed comments on the contents 
of the preliminary version of this paper. 
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Figure 1 . Label of some alcoves and Weyl chambers 



2. Notation and preliminaries 



We shall generally use the notation from |AKj . In this section we 
recall the main players and we introduce some specific notations for the 
(j2-case we are dealing with. Bits of further notations are introduced 
as we go along and we refer to loc. cit. for any unexplained notation. 



In the algebraic group G of type G2 we fix a maximal torus T and 
we let R denote the corresponding root system. We choose a set of 
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positive roots R^ and a and /3 with /3 long denote the two simple roots 
in R^. Then B will denote the Borel subgroup corresponding to — i?"*". 
The line bundles on the flag variety G/B are induced by the characters 
of B. We let X denote the character group of B, and for A G X we 
use the abbreviation H^{X) for the i-th cohomology group of the line 
bundle on G/B induced by A. 

Inside X we have the set of dominant weights X"*" = {A G X | 
(A, 7^) > for all 7 G i?"*"}. The Weyl group W is the group generated 
by the reflections in the walls of X"*", i.e., by the two elements s = Sa 
and t = sp. The bottom alcove in X"*" is Ai = {A G X+ | (A + p, 7^) < 
p for all 7 G R^}. Here p = 5a + 3/3 is half the sum of the positive 
roots. Note that G Ai because of our assumption p > 5. The affine 
Weyl group Wp is the group generated by the reflections in the 3 walls 
of Ai. The alcoves in X are the the mirror images of Ai under Wp via 
the dot action (i.e., the action with origin — p). 

Recall that as a consequence of the strong linkage principle |A80aj 
all composition factors of a cohomology module H^{\) have (dominant) 
highest weights in Wp ■ A. Moreover, if for A,p G Ai we denote by T^ 
the translation functor from the A-block to the p-block then we have 

T^H\w ■\) = H\w p) for all A G F, p G F. (2.1) 

Here F denotes a non-0-facette of Ai, i.e., F is Ai itself or one of 
its walls. By F we denote the closure of F. We note that any A G 
X belonging to a 0-facette must be an element of — p + pX by our 
assumption p > 5. 

By (2.1) we immediately deduce that if a cohomology module van- 
ishes for some weight in a facette (in our case an alcove or a wall) 
then the same cohomology vanishes at any other weight in the closure 
of that facette. More generally, this translation principle allows us to 
determine the composition factors of the cohomology of a weight in the 
closure of a facette once we have the corresponding information for a 
single weight in the facette. Therefore we abuse notation and some- 
times write H^{F) for a cohomology group of a (non-specified) weight 
in the facette F. 

We enumerate the first few alcoves in X"*" as in [H], see Fig. 1. The 
alcove containing the number i is then denoted A^. If Ai and Aj share 
a wall we denote this common wall by Fi/j. In Fig. 1 we have also 
given names to some of the Weyl chambers that we shall need often. 
We have set x = st,y = ts,z = sy, and w = tx. 
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If V is any element of W then we set A\ = v ■ Ai and similarly 
F^',- = V ■ Fijj. By the chamber v we will mean the chamber v ■ X"*". 

Recall that -^"(A) 7^ if and only if A G X+. Moreover, Kempf's 
vanishing theorem says that H^{X) = for all i > when A G X"*". 
Serre duality implies that i:f^(A)* ~ H^-\-\ - 2p) for all A G X. 

In characteristic zero the vanishing behavior of line bundle cohomol- 
ogy (on any flag variety) is determined by Bott's theorem [B] which 
says that H^{\) 7^ if and only if A G w • X"*" for some w &W oi length 
i. By semi-continuity (or use the universal coefficient theorem in (3.2) 
below) we have that the dimension of a cohomology module in charac- 
teristic p > is at least equal to the dimension of the corresponding 
cohomology module in characteristic 0. Hence we have H^{X) 7^ in 
all chambers w ■ X"*" where w &W has length i. In the following sec- 
tions we will therefore concentrate to the "extra" cohomology, i.e., on 
H^{X)^s or H\A)'s for A or A lying outside these chambers. 

In [A 79] the first author obtained the complete vanishing description 
for H^{X) and (via Serre duality) H^^ '~^(A), see also the alternative 
formulations in Corollary 2.7 and Remark 2.8 of [A07j . When A belongs 
to the lowest p^-alcoves we have in Fig. 2 illustrated this result in our 
case by entering 1 (resp. 5) on the alcoves where extra H^- (resp. H^-) 
occur. 

The above remarks mean that the cohomology modules we have left 
to describe are H''{X) with 2 < i < 4. We deal with these after first 
describing the methods we use in the next section. 



3. Methods 

In this section we will describe the methods we use to determine the 
vanishing or non- vanishing of a cohomology module if* (A). The results 
may all be found in the literature and are slight variations of the 6 step 
program sketched in |AKl Appendix B]. They all apply in the case of 
a general fiag variety although we have not always chosen to formulate 
them in their most general versions. Our aim has been to gear them 
towards the G'2-problem of the present paper. 

We set for each n > 1 

P„ = {A G X I (A + p, 7"^) < p" for all 7 G R}. 
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This is the union over all Weyl chambers of their lowest p"-alcoves. 
When A G Pi the cohomology H*{X) behaves according to Bott's the- 
orem in characteristic 0, see |A80aj . We shall use this to get our first 
results for weights in P2. 

We call a weight u & X special if there exists A G X such that 
u = p ■ X. Here we have extended the "dot notation" to include also 
multiplication by p on X, i.e., p- X = p{X + p) — p. Thus u is special iff 
p E p ■ (— p) + pX. Note that A G Pi if and only if p • A G P2. Also we 
say A is singular iff (A + p, 7^) = for some 7 G P, in which case p ■ X 
is singular and vice versa. Otherwise we say A is regular. Likewise p- 
preserves the individual Weyl chambers. 

Setting Xp = {A G X+ | (A, 7^) < p for all simple roots 7} we then 
have the following proposition, cf. Step 1 in |AKt Appendix B] 

Proposition 3.1. Let v he a special point. Then 

i) If v E P2 and is singular then P*(z/) = for all z G N. 
ii) // V is regular and w is the element in W for which w ■ u E X+ 
then P'("')(z/ ±X)^0 for all X e Xp. 



Proof, i) is a consequence of the isomorphism W{p-X) ~ H^{X)^^^0Stp, 
see [MHbl Theorem 2.5] (vahd for all A G X,i G N). Here Stp = 
H^{{p—l)p) is the Steinberg module. Moreover, ii) can also be deduced 
from this result, see |A07| Corollary 2.7]. D 

Remark 3.1. More generally, if A G X with P*(A) 7^ 0, then V/i G 
p*" ■ A ± Xpr with r G N, H\p) ^ 0. 

Proof. \i pef -X- Xpr, this is [AOTl Prop.2.6]. If p G f ■ X + Xp,-, 
we argue as in [A071 2.7]: by the Serre-duality P'^"*(— A — 2p) 7^ 0. 
Then \/r] e Xpr, y^ H^'^p'' ■ {-X - 2p) - r]). It follows from the 
Serre-duality again that 

y^ P'(-(/ ■ (-A - 2p) -T])- 2p) = H\p' -X + T]). 

n 



We shall also need the following proposition obtained by the trans- 
lation functors mentioned in Section2 . Let A be an alcove, s the 
reflection in one of the walls of A and denote by 6s the corresponding 
wall crossing functor. We will denote by As the alcove adjacent to A 
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over the s-wall. li As > A then we have the long exact sequence [A811 
Proposition 2.1] 

y H\A) -^ 0sH\A) -^ H\As) -^ ■ ■ ■ . (3.1) 

Then part i) of the following is (2.1). 

Proposition 3.2. i) Let F C X be a facette._ If W{\) = for 

some A G -F then -ff*(/u) = for all ^ & F. 
ii) Let s be the reflection in a wall of an alcove A and suppose 
As > A. Assume H'+^{A) = 0. 

a) If H\A) = then also H\As) = 0; 

b) If F denotes the common wall of A and As then W{F) ^ 
if and only if H\A) ^ ^ H\As). 

iii) Suppose for some alcove A we have W^^{A') = for all alcoves 
which are obtained from A by a sequence of alcoves A = Ai < 
A2 < ■ ■ ■ < Ar = A' with v4j+i = AiSi for some reflection Si 
in a wall of Ai, i = 1, 2, ■ ■ ■ ,r — 1. // H\A) = then also 
H\A') = for all such A'. 



Proof. To prove ii) we observe that our assumption here makes the last 
displayed map in (3.1) a surjection. This gives immediately part a). To 
obtain part b) we observe 9s = T^oT^ with X E A and fi E F. We have 
T^H\A) ~ H\F) ~ T^H\As) which gives the "only if part. The 
"if part follows from the surjection OsH\A) ~ T^H\F) -^ H\As). 
Finally iii) is obtained by a simple iteration of iia). D 



Exploring the translation functors further one obtains (cf. Step 3 in 
[AKl Appendix B] 

Proposition 3.3. Let u E X be a special point and denote by W^ the 
stabilizer of u in Wp. Assume H^{v) = for some j G N. 

i) // H^+'^{A) = for all alcoves A with u eA, then W{v + p + 

A^) = 0. 
ii) If in addition to the assumptions in i) we have H^^^{i' + X) = 

for all XeX+ then also H{{u + A) = for all X E X+ . 
iii) Let F be a facette with u eF. IfW+^{wF) = for allw E W^ 
and F"*" is maximal in {w ■ -F}«,evK^, then H^{F~^) = 0. 



Proof. Note that W,^ ~ W. Consider first iii). By translating from the 
z/-block to the F-block we obtain H^{F^) = 0, see |A81l Theorem 4.3]. 
Then i) is the special case where F is an alcove. Finally ii) follows from 
i) and Proposition 3. 2. iii). D 
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Let Zp be Z localized at the prime p and denote for A G X and i G N 
by H^ (A) the analogue of -ff*(A) over Zp. Then by "the universal 
coefficients theorem" we get the short exact sequence 



^ HliX) k -> W{X) ^ Tort'' (//^+^ (A), A;) ^ 0. (3.2) 

This gives us the following result 

Proposition 3.4. Suppose A G X (ioes not belong to a chamber w ■ X^ 
where w e W has length i + 1. If H'+^{\) = and H'+\\) j^ then 
also H\\) ^ 0. 



Proof. This is Step 4 ii) in |AK[ Appendix B] which also contains the 
easy proof. D 



The following proposition is a combination of Step 5 and 6 in |AK|. 
Appendix B]. 

Proposition 3.5. Let 7 be an arbitrary simple root with the associated 
reflection s^, and let X ^ X also be arbitrary, i.e., not necessarily in 

i) /f < (A + p, 7^) = sp^ for some s,m & "H with s < p, then 

H'+\s.f- A) ~/7^(A). 
ii) If ap < (A + p, 7^) < (a + l)p for some < a < p then we have 
the following two long exact sequences 

> W+\s^ ■ A) ^ H\\) -^ H'+\V) ^■■■ 

and 

> W+\C) -^ W+\V) -^ H\Q) ^■■■ 

where C and Q both have weights s^-X+p'y, s^-A+2p7, ■ ■ ■ ,s^- A+ 
apj, each occurring with multiplicity 1. 
iii) Suppose A satisfies the inequalities in ii). If H''^^{s^ ■ A) has 
a composition factor L{fi) which is not a composition factor 
of any of the modules H^{s^ ■ A + rp'y) for j G {i,i — 1} and 
1 < r < a then L{n) must be a composition factor of II^{X) 
forcing this module to be non-zero. 



Proof, i) and ii) are parts of the ingredients in the proof of the strong 
linkage principle |A80a] . iii) is an immediate consequence of ii). D 



4. H^ 



4.1. H'^ in the chambers t,y and w. First we apply Proposition 3.1. 
For each special point v in the chamber tz we have -ff^(z/ + A) 7^ for all 
A G Xp. This gives non- vanishing W^ in the alcoves marked 4 in Fig. 3. 
Proposition 3.2.iib) then implies that H'^ is also non-zero on the walls 
between these alcoves (we see from Figure 2 that the necessary H^- 
vanishing needed to apply Proposition 3.2.iib) is indeed satisfied). By 
Proposition 3.2.iii) we get also non-vanishing W^ in the alcoves marked 
o, and via Proposition 3.2.iib) also on the walls between them. 

By Proposition 3.1.i) we have vanishing cohomology at all special 
points on the wall between chamber w and chamber tz. Proposition 
3.3.i) then gives vanishing H* in the alcoves marked x in Fig. 3 (for 
alcove Ai^ we apply Proposition 3.3.iii) with F equal to its longest 
wall and then apply Proposition 3.2.ii). For the other alcoves we apply 
Proposition 3.2.i) to the alcove itself). Note that the short wall does 
not have a special point in its closure! 

Now Proposition 3.2.iii) allows us to deduce iJ'^-vanishing in all al- 
coves above the alcoves marked x (we say that an alcove A' is above 
the alcove A iff there is a sequence of adjacent alcoves A < Asi < 
AsiS2 < ■ ■ ■ < AsiS2 . . . Sr = A' as in Proposition 3.2.iii)). 

Proposition 3.4 gives H\A'^) ^ because H^{AX) ^ 0. The se- 
quence (3.1) gives us an exact sequence 

H\AV^) -^ H\Al) -^ e,H^{A^). 

Here the last term vanishes because H^ vanishes on the mid-sized wall 
(the s-wall) of A"^. On the other hand, the middle term is non- zero so 
that iJ^(A^) 7^ 0. An analogous argument gives i^*^(A^) ^ and then 
by Proposition 3.2.iia) also H'^iA^) ^ 0. 

li A = Af and A' = Af, then H\Fa^/a) + ^ H\Fa^/a') by 
Proposition 3.2.iib). 



Consider now F^,^. The sequences in Proposition 3.5.ii) are in this 
case 

H\F^/,) ^ H\V) ^ H\F^Jl,) 

and 

H\C) -^ H\V) -^ H\Q) -^ H%C). 

The last term in the first sequence vanishes. In the second sequence the 
weights of Q and C are F^J"^ + pa and Fgjg + 2pa. We have H'^F^J"^ + 
pa) = by Proposition 3.5.i) whereas H^{F^f^ + 2pa) = = H^{F^J'^ + 



2pa) by the results in Section 2. It follows that H^{V) ~ H^{Q) — 
H^i^6/8 + 2P«) 7^ and hence H^{F^/^) ^ 0. Then by Proposition 
3.2.i) we have also H^{A^) ^ 0. 

Applying the same method we get H'^lF^ij) = 0. The difference here 
is that F^?j + 2pa = F^,^ has vanishing H'^. In fact, weights on the wall 
F3/4 are minimal in X"*" (with respect to the strong linkage relation) 
so that if* (Fg,^), V G W, behaves as in characteristic 0, see |A80aj . 
Moreover, H^{F^)"j) = and the sequences corresponding to the above 
imply the claimed vanishing of H'^{F^,j). Hence by Proposition 3.2.iib) 
we get H\A^) = 0. 



li A = A^l , there are exact sequences 



H\FX^,^) ^ H\Fa^ja) -> H\V) ^ H\FX. 



and 

H\Q) -^ H\C) -^ H\V) -^ H\Q) 

with C = Q = FX-^ij^ + pa. As F^^j^ + pa e^i, H\FX^/a + pa) ^ Q 
iff i = 5. As H^{Fa^/a) = = H%Fa^/a), it follows that H\Fa^ /a) ^ 
H^{V) ~ if 5(C) = 'h\FX^j^ + pa) ^ 0. 

As Ai is the bottom dominant alcove, by the strong linkage principle 
again H\A'^) = 0. 

In Fig. 3 we have marked on the alcoves Af, i = 3,4,5,6 to 
indicate that they have non-vanishing H'^ whereas we have marked ♦ 
on Af for i = 1,7 because here we have vanishing H^. By Proposition 
3.2.iii) we then have vanishing H^ also for all alcoves above either A'^ 
or A^. 

This completes the description of the vanishing behavior of H^ in the 
intersection of P2 with the chambers t,y and w, and on the facettes 
between them by Proposition 3.2.iib). In particular, we record "ex- 
ceptional" vanishing H^{F^,^) = in contrast to a statement in |A81|. 
Section 5], cf. also [H], by marking x on the wall. 

4.2. if^ in the chambers s,x and z. We proceed exactly as above 
and record the results in Fig. 4. First we get the non-vanishing of 
H^ in the alcoves marked 4 and then in the alcoves marked with o. 
By Proposition 3.2.iib) H'^ is also non-zero on the facettes between 
these alcoves. Next we locate those special points on the wall between 
chamber z and chamber sw at which if ^ vanishes and where moreover 
the method in Proposition 3.3 ensures vanishing in the cones above 
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them. We indicate this by marking x on the bottom alcoves in these 
cones. As H^{A^) ^ 0, i = 10, 11, 12, Proposition 3.4 gives us that H'^ 
is non-zero in these alcoves. Then Proposition 3.2.ii) ensures that H"^ is 
likewise non-zero in the adjacent alcoves corresponding to i = 13, 14, 8 
and 9. Non-vanishing in Ag in turn implies for the same reason non- 
vanishing also in Aj and Ag. We have marked all these alcoves with () 
in Fig. 4. 



?tz 



We now observe that by Proposition 3.5 we have H^i^F^^,^^ ^ H^^^{Fl-^i-^^j 
for j > 4. The arguments based on Proposition 3.2 then give 
H^{Al^) ~ H^(AIj^) so that by translating to the wall we get H'^(F^^,-^^) ~ 
-^^(-^8/11) ~ ^- ^^ follows via Proposition 3.2.iib) that H"^ vanishes on 
Af^ (which we therefore mark with ♦) as well as on all alcoves above 
it (Proposition 3.2.iii)). 



Consider now the exact sequences coming from Proposition 3.5.ii) 

u/s) 



H\V) ^ H'{Flt>,) -^ H\FU) ^ H\V) 



and 

H\C) -^ H\V) -^ H\Q) -^ H\C) -^ H\V) -^ H\Q) 

z 
1/8 



with C = Q = Flli^ + pI3- As i^n/g + p/3 is a W^-conjugate of the 



long wall of Ai we have H'{F^'y^ + p/S) = 0. It follows that H*{V) = 
= H^{V), and hence H^^F^y^) ^ H\Fllj^) = 0. Then H^A^) ~ 
H^{Ali) by Proposition 3.2. Translating to the wall we obtain H'^{F^,q) ~ 
if^(F{3,^5) = 0, and hence H*{Aq) = again by by Proposition 
3.2. Hence we mark Ag with ♦ and also we get vanishing in all al- 
coves above it. We see likewise H*{F^^,-^^) ~ 77^(^*4,-^2) = while 
H\F^o/9) ^ H\F{^/g) ^ 0. 

The sequences for F^,^ analogous to the ones above give H'^i^F^:^) ~ 
H^{F^^,.^) 7^ (we use for this the observation that H*{F^^,^+p(3) = 0). 
Hence also H'^^Al) 7^ and we have therefore equipped this alcove 
with a in Fig. 4. 

When i = 1, 2, 3 we have (Proposition 3.5.i)) H^iAf) ~ //^(Af ) = 
giving rise to ♦ marking of these alcoves. 

Finally, we examine H'^{F^q^^^), H^iF^^^^^), and H^iF^zjAU.). Put 
for simplicity F = F^z ^^tw. There are exact sequences 

H\V) -^ H\t ■ F) -^ H\F) 

and 

H\C) -^ H\V) -^ H\Q) 
11 



with C = Q = t ■ F + pP. Another apphcation of Proposition 3.5.i) 
gives H\t-F + p(5) ~ H'+\t-{{t-F+p^)) = unless i = 5. It follows 
that H^{V) = 0, and hence that H^{F) surjects onto H^{t ■ F) ^ 0. 
Likewise H\F,^,^,,) ^ H'iF^^,/,,) = = H^iF^^,/,,) ^ H\F^,/,,). 

This completes the the description of the vanishing behavior of if ^ 
in the intersection of P2 with the chambers z,x and s, and also on 
the facettes between them. We record H'^{F^^,^r^) = H'^i^F^^,^^) = 

^^(^{0/12) = ^^(^A/ia) = 0' see again [MI] and [H]. 



5. H^ 



5.1. H^ in the chambers t and y. The same methods as used for 
H^ above give non- vanishing of H^ in the alcoves marked 3 and o, and 
vanishing in the alcoves marked x or ♦ and all alcoves above such, see 
Fig. 5. Moreover, we also have H^{A\) = so that we can mark this 
alcove by a ♦. As before we have vanishing H^ in alcoves above it. 

Our if*^-results in Section 4 give non-vanishing if^ in the alcoves 
A^ for i = 20, 22, 24. By Proposition 3.4 this implies non-vanishing 
of H^ in the same alcoves and then by the exact sequence (3.1) also 
first in the alcove A\q as H'^^F^^,^^) = 0, and then by Proposition 
3.2.iia) in A^, i = 9, 10, 12, 14, 17. We have therefore marked all these 
alcoves 0- Proposition 3.4 gives non- vanishing of H^{F^2/2o^' -^^(-^24/20) 
and H^{Fj^y ja-w ) among the walls of the alcoves A\ for i = 20, 22, 24. 
Moreover, 

H%%26) - ^'(^22/25) by Proposition 3.5.i) 

7^ as we have seen in §4, 
H\Fy,/J ^ H\F^y,,) by Proposition 3.5.i) 

= as we will see in 6.2, 

^'(^Io/it) - ^'(^20/17) by Proposition 3.5.i) 

= as we will see also 6.2, 
if3(F|,/3j ^ H\F^,/,,) by Proposition 3.5.i) 

7^ as we have seen in §4. 

The remaining walls in this chamber are now covered by Proposition 
3.2 iib). 

Consider then A^. By Proposition 3.5.ii) we have the exact sequences 

H\A'^) -^ H\V) ^ H\A^) 
12 



and 

By Proposition 3.5.i), in the first sequence H^{A^) ~ H^{Ai) and in 
the second sequence H^{A^) ~ H'^iA^) whereas H^{Al) ^ -^^(^5) = 0. 
Now by |A86] -^(^4) is a composition factor of H'^IA^) but not of 
H^A-r). We conclude that H^A^^) ^ 0. 

Likewise we have exact sequences 

and 

H^{Al) ^ H^{V) -^ H\Al) -^ H%Al) 

with H^{Aq) ~ H^~^{Aq) by Proposition 3.5.i), which vanishes unless 
j = 1. As H'^{Al) ~ H^{As) = by Proposition 3.5.i) again, we get 
H^{Aq) -» H^IaI). But the composition factors of H^{Aq) are L{Aq) 
and L{A5) while H^{F^/q) = as i/^(A|) = 0. As the translation 
to the Fg/g-wall annihilates neither L{Aq) nor ^(As), we must have 
H^[Ag) = (which we therefore marked ♦) and on all alcoves above 
it. 

This completes the determination of H^ in these chambers. We 
record H^iF^.^.^) = ^'(^'o/ir) = 0- 

5.2. H^ in the chambers s and x. Once again we get non-vanishing 
in the alcoves marked 3 and o as well as vanishing in the alcoves marked 
X and all alcoves above them. 

By Proposition 3.5.i) we get H^{Af) ~ H'^{Ai) = 0, so we have 
marked alcove Af with ♦ and have vanishing of H^ in all alcoves above 
it. On the other hand we get H^{A^) ~ if^(A^) 7^ 0. 

Weights on the facet F3/4 are minimal in X~^ (with respect to the 
strong linkage). Hence H^{Fy^) = by [A80aj . This gives vanishing 
of H^ first on Af (marked ♦ in Fig. 6) by Proposition 3.2.iib) and then 
on all alcoves above it by Proposition 3.2.iii). 

The /3-sequences from Proposition 3.5.ii) relative to A^ give H^{A^) ~ 
H'^IA^) 7^ 0. So we mark A^ and subsequently A^ by (> from the exact 
sequence (2.1) as if^(^g) = 0. The same sequences with respect to Af-^ 
give H^{Af^) ^ H^{Af^) = 0. We mark Af^ with ♦ and H^ vanishes 
on all alcoves above it. 

Note that iif^(y4|3) 7^ 0. By Proposition 3.4 this means that also 
i7^(ylf3) 7^ 0. The exact sequence (2.1) then gives non- vanishing H^ in 
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the adjacent alcove A21 as well as in the alcove Afg below it. We mark 
these alcoves in Fig. 6. 

By Proposition 3.5.i) we have H^F^y^^) ~ H^F^^j^^) ^ 0. Like- 
wise H^{F^^i2-^) ~ if^(F23/2i) which does not vanish as we will see in 
6.1. Also, as ^^{Fax jj^z ) ^ by Proposition 3.2.iib), we get from 
Proposition 3.4 that H^{FA-/Aii.,) ^ 0. 



Consider now F^^i^^. The exact /3-sequences from Proposition 3.5 
give 

H\F-,/,,) -> H\F^,/,,) ^ H\V) 

and 

H\C) -^ H\V) -^ H\Q). 

Here our if ^-results show that H'^{F^2/25) ~ ^ ^^^ ^J looking at 
the weights -F^ ^5 + pP and -F^/25 + 2p/3 of both C and Q we see 
that H^{C) = = H^iQ). Hence H*{V) = and it follows that 
7/^(^2^2/25) = 0. Therefore also //^(Afg) = by Proposition 3.2.ii) as 
-^^(^22) ~ 0- W^ mark this alcove ♦ in Fig. 6 and have vanishing H^ 
in all alcoves above it. 

We claim that H^{A22) 7^ 0. To see this consider the a-sequences in 
Proposition 3.5.ii) 

H\Al2) ^ H\V) ^ H'{A^22) ^ ^'(^22) 
and 

H\V) -^ H\Q) -^ H\C). 

The weights of C and Q are {A22 + jpcx \ j = 1,2,3}. We know 
H^{A22 + p<y) = 0. Also by Proposition 3.5.i) and by Kempf we have 
H^Ia^2 + 3pa) - H^it ■ (^22 + 3p«)) = = ^^(^22 + 2pa); 

H^{A^2 + 3pa) = H^iA^o) ^ H^{A2o) by Proposition 3.5.i) 
= by Kempf 
= H^{Ais) = H^{A22 + 2pa) by Kempf again. 

It follows that H^{C) = 0, and hence we obtain an epi H^{V) -^ 
H\Q). Likewise, as 7/^(^*2) ^ H\A22) = and as 7/^(^*2) ^ 
-^^^(^22) both by Proposition 3.5.i), we obtain an exact sequence H^{A22) — ^ 
H'^{V) — 7- H^{A22) — ^ 0. Moreover, we have an exact sequence 

H\Q') ^ H\A^22+PC^) ^ H\Q), 

where the weights of Q' are A22 + '2pa and A22 + Spa. As A22 + 3pa is 
non-dominant it follows that H^{Q') ~ H'^{A^2 + 2p«) = ^"(^is)- On 
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the other hand, the a-sequences in Proposition 3.5.ii) yields an exact 
sequence 

^ H\A^,,+2pa) ^ H\A^,,+pa) ^ H\s-iA^,,+pa)) -> H\A^,,+2pa) 

with i7°(Af2 + 2pa) = H^iAi3), H^{A%^ + 2pa) = i^H^is) = 0, and 
H^{s ■ {A22 +pa)) = H^{Ai4). Now we use the observation in Proposi- 
tion 3.6 that H^{Ai4) has composition factor L{Aq) while H'^lAi^) does 
not. It follows that H^{A22 +poi), and hence also H^{Q) and H'^{V) 
have composition factor L{Aq). As H^{A22) does not have composition 
factor L{Aq) again by [A86j . we conclude that L{Aq) is a composition 
factor of H^{A22) proving our claim. 

If p = 7, as Ai3, A\^ and A12 do not live in the Jantzen region, [A86j 
does not directly apply. One can, however, compute the multiplicity 
of L{A^) in E\A^-i),E^{Ax^),E^{A22) using the dT-version of the 
Lusztig conjecture, G\ the Frobenius kernel of G, which holds thanks 
to |JCJj . by recalling 

{E\A) : L{C)\ = J2MA) : L{u)][L{u') ® xi^^T^ : L{C)], 
uex 

where u = u^ + pv^ with Uq G Xp and z/^ G X, x{^^) = l^ii^^Y^^i'^^)^ 
and [V(A) : L{h')] is the multiplicity of the GiT-irreducible L(i/) of 
highest weight u in the GiT-composition series of V(A) = ind^^ rp(A) 
with Bi the Frobenius kernel B. 

It follows from the exact sequence (2.1) that also the alcoves Af^, 
Afg, Af^, A21, and Afg have non- vanishing E^ and they have been 
marked accordingly in Fig. 6. 



6. E^ 

6.1. E^ in the chamber t. Propositions 3.1-3 suffice to determine 
the vanishing behavior of E^ in this chamber, see Fig. 7. 



6.2. E^ in the chamber s. Propositions 3.1-3 again settle most al- 
coves in this chamber and they give E'^{Al) = 0. The only alcoves need- 
ing special considerations are Aj with j = 9, 10, 12, 14, 17, 18, 24, 31, 32, 33, 34, 35. 
We claim that E^ is non-zero in the first 10 of those but zero on the 
last two. To prove this, using the exact sequence (2.1) and Proposition 
3.2.na), it is enough to show that if^(A^g) 7^ 7^ E'^^A^^) whereas 
ff2(^sj _ 0. See Fig. 8. 
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Figure 2. Extra H^- and i/^-nonvanishing 



Consider the /3-sequences for A 



s 
18 



H^iV) ^ H\A%) -^ H\A 



is) 



and 



H\C) ^ H^{V) ^ H\Q). 



By our results obtained in 5.1 we have H^{A\^) ^ 0. Moreover, C = 
Q = A\g + pf3 = Aio is dominant so that W{C) = W{Q) = for all 
j > 0. The non-vanishing of H'^{Alg) follows. 
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Figure 3. Extra if^-nonvanishing in the chambers t,y 
and w 



To see that -^^(^33) is non-zero, we consider the a-sequences 



H\A,,) ^ H\V) ^ H\A, 



33) 



and 



H\V) -^ H\Q) -^ H^{C). 

The weights of C and Q are A33 + pa = A2Q and A33 + 2pa = A^-j. It 
follows that H'^{C) = by Proposition 3.5.i) and by Kempf, and that 
we have an exact sequence 



H\Q) ^ H\A27) ^ H\A 



26) 



Here the last module is isomorphic to H^{A2q) by Proposition 3.5.i) 
again. According to |A86] L{Aq) is a composition factor of H^{A27) 
but not oi H^{A2e). We conclude that L{Aq) is therefore a composition 
factor of H^{Q) and hence also of H^{V). As it is not ( |A86] again) 
a composition factor of H^i^A^s) the first sequence gives the claim. 
Although A2Q, A27, A33 do not live in the Jantzen region for p = 7, one 
can compute as in 5.2 the relevant multiplicities of L{Aq). 
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Figure 4. Extra if'^-nonvanishing in the chambers s^x 
and z 



Finally, to check the vanishing of H'^i^A^^ we look at the /3-sequences 
E\Al,) ^ E\Al,) ^ E\V) 
and 

Here -^^(^34) = by 4.1 and considering the weights A34 + 2p/3 = Aig 
and A^4 +J9/3 = A\^, we get //^(q) = q = iJ3(c') from 5.1 and 4.1. 

18 



>J 


<{ 


> 


^ 


\ 




c^ 






i? 


^ 


s 


\ 


J? 




<^ 


^ 


1; 


$ 








^ 


\ 


5? 


\ 


2S 


5j 






\ 


p 


^ 


rp> 




\/3 
/3> 




\/3 
/a 


^ 


<A3 

/A 

As 




AA 
/A 
A'^s 


r^ 


<^ 


7\ 


v> 




\3> 
3\\ 




^N. 3 

3>r 

<3\3 
3^A 


73> 


AC? 


\/3 

A> 
AA 


^ 


\/3 

A> 

/^3 


A> 
AV 

3^A 


^ 




> 


p 


;^ 




>J 




>J 


A 


^ 


A 


<7 


% 




^ 


% 


\ 


1? 






^ 




J^ 


% 


5j 


;| 


\ 






\ 


% 


j^ 










>? 


\ 


$ 




$ 



Figure 5. Extra if^.^onvanishing in the chambers t 
and y 



7. The remaining cases; what should we name? 



The above results together with Serre duahty describe the vanishing 
behavior of all line bundle cohoniology corresponding to weights in the 
lowest p^ alcoves. The results are collected in Figure 12 below (identical 
to Fig. 3 in [Ml §8]). 
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Figure 6. Extra if3-nonvanishing in the chambers s 
and X 



8. Higher alcoves 



In |A81] (5.7) the first author formulated a principle which he hoped 
would allow to extrapolate the vanishing behavior of W in the higher 
alcoves from those in the closure P2 of the union of the lowest ]?- 
alcoves. Properly formulated (see [AKJ Appendix B) this principle 
says 
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Figure 7. Extra i/^-nonvanishing in the chamber t 



For an arbitrary weight A we have H'^{X) ^ if and only if there 
exists X G P2 with H'^{x) 7^ such that A G p" ■ x ± Xpn for some n. 



The 'if part' of this principle follows from Remark 3.1. The 'only if 
part' has only been checked (using the methods above) for types A2 
and B2 (and in general for i = 1 and i = N—1). The following example 
shows that it fails for type G2- 
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Figure 8. Extra i/^-nonvanishing in the chamber s 



Consider A = {2p^ — p)ua — P^oo/s + ^2- This alcove is a subset 
oi p ■ u + Xp and oi p ■ u' — Xp where u = 2{p — l)uJa — {p + ^)^p 
and u' = u + p. Note that u G F^,^ and u' G F^,^ so that by our 
results in Section 4.1 we have IF^ii^) = = H'^{v'). Therefore the 
principle above predicts vanishing of H'^{A). However, if A' is the 
alcove obtained by reflecting A in its mid-sized wall then H^{A') ^ 
(e.g., because A' d p^ • (— Sw/j) + Xp2). It follows from Proposition 3.4 
that then also H'^{A) ^ and hence (5.7) fails for weights in A. Using 
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Figure 9. Extra cohomology non- vanishing for p > 11 



the arguments in Proposition 3.2.ii) we find many (17 for p = 7) other 
alcoves for which (5.7) fails. 



We shall illustrate this result by taking p = 7. Enter first 5's in the 
alcoves with extra if^-non-vanishing in Fig. 10. Then enter 4's where 
we have extra if^-non-vanishing in P2 from §4. Also enter 4's in the 
alcoves belonging to those p^-alcoves where Proposition 3.1 gives extra 
non- vanishing H"^. 
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Let Af denote the p^-alcove of type i and let A^'^ = v ■ A'f, v G W. 
By Proposition 3.4 we have non-vanishing H"^ in all of A^'^ and we 
therefore mark all alcoves here which are not already equipped with 
a 4 by 0- Let A' be the top alcove of A^'^ and let A = A't. From 
(3.1) there is an exact sequence H^{A) -> H^{A') -^ OtH^iA). As 
H^{A') ^ = 9tH^{A), we get H^{A) ^ 0. Mark thus A with 0, and 
also all alcoves in A^'^ below A by Proposition 3.2.iia) (unless such 
an alcove already contain a 4). By Proposition 3.3 we can mark 2 
translates of Ai with x, and then H'^ vanishes on all alcoves above 
either of those; the relevant special points are p ■ {pooa — {p— 1)1^/3) and 
p-{{p+'i)uJa—poJii) with both pc<;Q, — (p—l)c(;/3 and (p+3)co'q,— pcu/3 G F^i^- 

Now 

A = {2p- l)puJa - P^OOp + A2 

C {p . ((2p - 2)u^ -{p+ l)cup) + Xp}n 

{p ■ {{2p - \)uJa_ - pUJfi) - Xp} 

with (2p - 2)uja - (p + l)w/3 e -^3^/4 and (2p - \)oJa - P^ji G Fj'/g. But 
our results in P2 say that H\F^/^) = = H\F^/^). Also, 

A^{p^. {-2ujp) + Xp2} n {p^ ■ {u^ - Up) - Xp2}n 

nr>3 {{/ ■ i-^a - 2uJp) + Xpr} n {p' ■ i-CUp) - Xpr}} 

with —2uJi3 G F4j»o/^tz, Wq, — w^ G -^Ai/a*) ~^/3 ^ -^Ai/a* and — w^ — 
2c<;^ G F^^o /^=»o, showing failure of the hoped-for principle. Moreover, 
H*{A - pua) 7^ with A - pua C {p • ((2p - 3)a;« - (p + l)^/?) + 
Xp} n{p-{{2p- 2)uJa - poop) - Xp} and (2p - 3)a;« - (p + l)a;;3 e F^^^, 
{2p — 2)ua — poop G F^i^. Likewise H'^{A — 2pUa) 7^ with A — 2puja C 
{p ■ {{2p - A)ua -ip + l)ojp) + Xp} n {p ■ {{2p - 3)ua - pojp) - Xp} and 
(2p-4)w^-(p+l)w^ G Fg'^^, (2p-3)wc«-pcj/3 e Fa»m^, ^^(^a»m^) = 
0. Also, 

A — pU)a,A — 2pUJa ^ {P^ ■ ( — 2w^) + Xp2} n {p^ ■ (cUa — tUp) — Xp2}n 
nr>3 W ■ i-^a - 2Up) + Xpr} n {/ ■ i-OJp) - Xpr}}. 

Thus all the alcoves in A^'^ above Ai + {2p—3)pUa—p'^ujp = {A—2pUa)s 
with s the short wall, 18 of them together in this p = 7 case, exhibit 
failure of the principle. 

On the other hand, let ui = p ■ ((2p — 5)ua — poJp). As {2p — b)uja — 
pup G A^ with H'^{A^) 7^ 0, the alcoves in vi + Xp are marked by 
the 'if part of the principle. If U2 = Ui— pu^ = p • {{2p — Q)ua — P^p), 
{2p — Q)ua —poJp G F^™/A«^ with H'^{F^m,y^tz) ^ 0. Thus the alcoves in 
v-2, + Xp are also marked with (>. 
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Figure 10. Beyond p^-alcoves for p = 7 



9. The quantum case 



As in the introduction we let Ug denote the quantum group of type 
6*2 at a root of unity g G C. More precisely, we start with the generic 
quantum group U^ of type G2 over the field Q(f), v being an inde- 
terminate. The generators in Uy are denoted Fi,Ei,Kf, i = 1,2. We 
choose the enumeration such that the indices 1 and 2 correspond in the 
notation of the previous sections to the short simple root a and the 
long simple root /3, respectively. Then we set A = 1j\v,v~'^\ and let Ua 
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denote the Lusztig A-iorm in U^, see e.g. [L90j . Considering C as an 
A-algebra via the speciahzation v ^-^ q we then define Ug = Ua ®a C 
This construction makes sense for all non-zero g G C but for our pur- 
pose the interesting case is when g is a root of unity. So for us q E C 
will always denote a primitive root of 1 of order I > 5 and we assume 
{1,6) = I. 



V 



We have a triangular decomposition U^ = U~U^U^ with U~ , U\ 
and [/+, being the subalgebras generated by the F's, -fC's, and -E"s, 
respectively. There is a corresponding decomposition Uq = UgUgUq. 
Following the convention in |APW91j we set Bg = U~U^ and denote 
by H^ the induction functor from integrable i?q-modules to integrable 
f/g-modules. This is a left exact functor and we denote its i-th right 
derived functor by if*, i e N. Setting X = Z^ each X E X defines a 
1-dimensional i?q-module which we also denote A and then the if*(A)'s 
are quantized counterparts of the line bundle cohomology studied in 
the previous sections. For details we refer to [APW91] . 

Now all the methods and techniques from Sections 2-3 apply in this 
case as well. In addition to [APWQl] we refer to |APW92j . [AWj which 
we need for non prime power /, and |A03] . These allow us to describe 
completely the vanishing behavior for -ffg(A) for alH G N and all A G X. 
The results are completely the same as those described in Figures 2-11 
with the only difference being that p should be replaced by / and there 
are no upper bounds on A (i.e., the condition A G P2 is not relevant in 
the quantum case). So we can summarize this as 

Proposition 9.1. The vanishing behavior of the cohomology modules 
Hg{X) of the quantum algebra is given by Fig. 9 with all alcoves being 
l-alcoves and with the figure extended to the whole plane. 
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